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ABSTRACT 



Common factors among high-performing distance learning (DL) 
programs were examined through case studies at 9 further education colleges 
and 2 nonsector organizations in the United Kingdom and a backup survey of a 
sample of 50 distance learners at 5 of the colleges. The study methodology 
incorporated numerous principles of process benchmarking. The study 
established that, in terms of planning, monitoring, and managing provision, 
good practice for DL has much in common with good practice for any program 
delivered through other modes of attendance. The data strongly suggested a 
correlation between meeting quality criteria in the planning, resourcing, and 
supporting of DL programs and the successful outcomes achieved by learners 
and the DL organization itself. The following factors were identified as 
critical success factors for DL: (1) integration of program planning, 

monitoring, management, and resources; (2) a good understanding of the DL 
operation's costs; (3) tutor and other contact with learners; (4) pre-entry 
guidance; (5) well-established procedures for selecting learning materials 
and monitoring their use; and (6) procedures to obtain customer feedback and 
a process to incorporate feedback survey results into future program 
planning. (The DL provider questionnaire and background summary are appended, 
along with the student questionnaire. Contains 19 references.) (MN) 
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Odtadtotoim 



Distance learning’s unique capability to meet the individual needs of a 
wide range of learners and thus to widen access to learning makes it an 
extremely important strand of provision in the new post-16 sector. Only 
quality distance learning can do this. 

Employers need responsive training that meets the needs of individual 
employees or small groups, that is cost effective to provide and achieves 
the desired outcomes. High quality distance learning is a successful way 
to provide such training. 

This report is the result of an investigation using case studies at nine 
colleges and two non-sector organisations and a back-up survey of a 
sample of students at five of the colleges. 
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Good practice for distance learning has much in common with good 
practice for any programme delivered through other modes of attendance, 
in terms of planning, monitoring and managing the provision. 



Flam) roemig 

Our research strongly suggests that there is a correlation between meeting 
quality criteria in the planning, resourcing and supporting of distance 
learning programmes and the successful outcomes that the learners and 
the organisation themselves achieve. A number of providers had high 
retention rates for distance learning. These were sometimes but not 
always accompanied by high achievement rates. We believe this merits 
further investigation. 



Management oimteg ration 

Colleges often seem to find it difficult to integrate the management of 
distance learning programmes into the main college planning, funding and 
management cycles. However, there were college providers who had long- 
standing and successful distance learning operations integrated into their 
main college processes, showing that integration is possible. Providers 
who achieved high quality outcomes regarded planning, monitoring, 
managing and resourcing as related not separate processes, and they 
operated them in an integrated way. This seems to be a critical success 
factor for a distance learning service. 



Data 

We had hoped to gather data to allow us to suggest metric benchmarks 
for distance learning provision but our initial research showed that data is 
not currently gathered and processed separately for distance learning so 
we could not proceed on this. Later we experienced some colleges’ 
difficulties in providing data on their distance learning provision and were 
glad we had not attempted to work on metric benchmarks at this stage. 

We recommend to the Learning and Skills Council (LSC) that it revises the 
method of collecting data to allow metric benchmarks to be set for 
distance learning as soon as possible. 



Most college providers did not have a clear idea of the costs of their 
distance learning operation. The problem did not appear to us to be so 
much with funding as with planning - only those who planned well knew 
what distance learning cost them. 



Tutor and other contact with the learner is a critical success factor for 
distance learning. Tutor contact in itself does not ensure success — more 
subtle characteristics about the support emerged as crucial (eg speed of 
initial contact, close monitoring of students’ progress etc). All providers 
recognised the importance of the tutor to their operation but there were 
marked differences between providers with successful outcomes and 
others in the rigour of selection and management procedures. 
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Pre-entry guidance is another vital component of successful distance 
learning provision. Only 46% of distance learners said they had received 
initial guidance on their choice of course but 91% of all those learners who 
had received guidance said it was helpful or very helpful. 



Providers with the most successful outcomes had procedures not only for 
selecting learning materials but for monitoring their use and using the 
feedback in their management and review processes. The quality of 
learning materials used was variable. Of all learners, 67% said the 
materials used were effective or very effective and 33% said the materials 
used were not at all effective or only reasonably effective. Only 28% of 
learners had been shown examples of the learning materials that they 
were going to use. Where learners had been shown materials, 76% of 
them considered it to be helpful or very helpful 



Most distance learners study almost entirely on their own. The contact with 
their tutor is usually by telephone or post. More than 90% of distance 
learners only ‘occasionally’ or ‘never’ use the facilities on offer to them 
such as computers or libraries, only 10% of them are in touch with other 
learners, and the majority of even that contact is only by telephone. The 
consequences of any poor practice are particularly exposed in distance 
learning programmes because the students cannot call on the social 
factors that can help to retain learners who are thinking of leaving an 
attendance-based course. 
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All college providers had procedures in place to obtain customer feedback; 
all had a customer charter (often the college charter) and a complaints 
procedure. Providers with the most successful outcomes were 
distinguished from others in the use they made of the survey results, the 
feedback from learners or the record of complaints. 

Other §ypp©ot for pr©voc3@r§ 

There is at present a tension between the providers’ wish to exploit the 
very varied potential range and design of distance learning courses to 
meet client need, and the funding bodies’ legitimate need to frame a 
definition of ‘distance learning’ to ensure quality in publicly funded distance 
learning provision. We believe that this tension may inhibit the expansion 
of distance learning. We suggest that the LSC considers providing a 
simple minimum specification for distance learning programmes, or a 
series of such specifications, together with guidance on a costing method, 
a revision of the collection of data for distance learning to standardise and 
simplify it, and recommendations on good practice. 
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Backgroyoid to to® stodly 



Open and distance learning are established methods. The best schemes 
have been very successful in meeting the needs of individual learners and 
in delivering high retention and achievement rates at a cost acceptable to 
the learner, or their employer, and to the providing organisation. However, 
the flexibility of time, pace and place of learning that the methods offer 
have also led to debate and confusion over the definitions of open and 
distance learning over many years. 

In turn this confusion has led to loose practice over the use of definitions 
of open and distance learning. Research by the National Extension 
College (NEC) in 1998 into the definitions of open and distance learning 
used in the FE sector exposed widespread operational confusion over 
their classification. One curriculum director summed it up by saying: 

Yes we do have definitions of open and distance learning but they 

mean different things to different people. 

What one college defined as part-time provision might be classed by 
another as open learning. Others considered some provision to be open 
learning that the then FEFC definition would classify as distance learning. 
The situation was further complicated by the funding advantages that 
some colleges perceived were to be gained by defining provision one way 
rather than another, with consequent anomalies reflected in some 
individualised student record (ISR) data. 

Also, some colleges, not necessarily those with prior experience as 
providers of open or distance learning, saw opportunities post 
incorporation either to offer open or distance learning themselves or to 
contract out parts or the whole of a distance learning operation. The poor 
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quality of much of this provision, as measured by retention and 
achievement rates, was highlighted in the FEFC national report from the 
Inspectorate on Open and Distance Learning (2000-01, referred to below 
as the FEFC report on ODL). Many providers of good quality distance 
learning, including the NEC, were concerned that the poor quality of some 
of this recent high-volume provision in a few specialist areas could 
damage the reputation of the method in the FE sector, and beyond. 

This combination of factors has resulted in a dearth of robust data about 
open and distance learning provision in the FE sector. There would be 
considerable consensus among experienced practitioners, whether in a 
particular college, corporate or private provider, on the prerequisites for 
quality open or distance learning provision. In recent times, however, there 
has not been any widely available study to provide comparative data for a 
metric or process benchmarking exercise. The FEFC report on ODL 
concluded with the recommendation (para. 76) that The funding council 
should develop and publish benchmarks for distance learning...’. 

The need for reliable comparable data has also become more urgent as 
providers everywhere consider and engage with the opportunities provided 
by e-learning. Quality standards for open or distance learning, the 
processes required to achieve those standards and the indicators needed 
to measure the success of the processes, are not likely to be identical for 
open and distance learning and e-learning, but all concerned recognise 
that the issues are often the same. Research into open and distance 
learning is relevant to the e-learning debate and vice versa. 

The study was set up as an independent piece of research jointly funded 
and managed by the Learning and Skills Development Agency (LSDA) 
and the NEC. 
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The study’s aim was to apply the complementary expertise of the LSDA 
and the NEC to the initial stage of work on research into open and 
distance learning in colleges. We decided to use the methods of process 
benchmarking whereby ‘benchmarking is a systematic method of 
improvement that utilises others’ good practice to improve your own 
processes’ (Owen 1999). We wanted to identify areas where it appeared 
from the study that there were common factors that contributed to quality, 
as the basis for the development of metric and process benchmarks for 
open and distance learning. 

Preliminary investigations in the FE sector quickly confirmed that, as the 
FEFC report on ODL had noted: ‘Open learning is harder to define than 
distance learning’. Colleges had very variable practice in the classification 
of provision as ‘open learning’ and, as a result, in some cases, they had 
potentially large areas of open learning provision. 

We judged that the potential scale and complexity of an investigation into 
open learning were beyond the capacity of this small study, although they 
clearly merited investigation, and we decided to restrict our research to 
distance learning. We adopted the definition of distance learning used in 
the FEFC Guidance on further education funding eligibility and rates, 
2001-2 (referred to below as Funding guidance) (para. 183) for distance 
learning programmes other than Ufi: 

Those on which learners study with specially prepared learning 
materials for their private study and are provided with active 
learner support, by suitably qualified staff, to enable them to 
successfully achieve the outcome identified in their learning 
agreement. This definition is intended to cover situations in which 
study is essentially home or work based and there is only 
occasional contact with the institution. 
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The outcomes of the study were: 

o identification of those aspects of the processes in distance learning 
provision that seem to enable good results to be achieved as the basis 
for pr@<£©§§ boondhiinniarks, based on an analysis of quantitative and 
qualitative data 

© the development of the coJIeg© questfomuroaiire as a research tool and 
as a ‘self-check’ tool to help the self-assessment reviews and action 
planning of providers offering distance learning 

o the slharoimg of goodl praofoc© found in the study. 
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Ten colleges in the FE sector were selected initially to be as 
representative a cross section as possible in terms of size, geographical 
location, scale of distance learning and experience of distance learning. 
We consulted the individualised student record (ISR) data and drew on the 
NEC’s considerable knowledge of distance learning in the sector to make 
the selection. In the event, nine participated. We aimed to include a 
majority of colleges where the evidence suggested that distance learning 
was operating successfully. The validity of our selection was checked with 
the National Institute of Adult Continuing Education (NIACE) and with the 
former FEFC lead inspector on open and distance learning. The retention 
and achievement data of those on the list was also screened by the LSDA. 

Desk research carried out in the initial stages of the project had highlighted 
some of the key issues that were likely to affect the provision of distance 
learning. These issues gave us useful guidance on the possible key areas 
within distance learning provision. 

The project involved a number of researchers talking to some very 
different colleges and other organisations. We decided to produce 
questionnaires in order to ensure that the information we obtained from 
each was comparable (see Appendix 1 ). The questionnaires were sent to 
the organisations to give them advance notice of what information would 
be requested along with enough time to gather relevant documentation. 

The questionnaire was produced using the same principles as those used 
in producing benchmarking questionnaires and followed the basic 
template below: 
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o What is the process? 

o Is there a documented description or flowchart of the process? 
o What process measurements do you use? 
o What are the current measurement results? 

© What aspects of the process work well? 
o What are the problems you have with the process? 
o What improvements have you made to the process? 
o Is there any other information you feel might be helpful to us? 

(Owen 1999) 



It was tested with staff not involved in the project before use. 

The questionnaire was then sent to each of the organisations taking part in 
the project in advance of the researcher’s visit. Colleges provided a 
considerable amount of information to support the answers they gave to 
the questions. 

To provide some comparison with the results from colleges and to take 
account of the new, wider post-16 sector, we also decided to include 
evidence from two non-college providers of distance learning. One was in 
the public service sector and the other was a commercial organisation. 
Both had been awarded the British Association for Open Learning (BAOL) 
Quality Mark for open and distance learning and were chosen with advice 
from BAOL that they were quality providers of distance rather than open 
learning. 

Each organisation in the survey was visited by a project member, or in 
some cases two, for 1 day. They met with key staff, including those who 
had strategic and operational responsibility for distance learning. 




17 



15 



We also wanted to validate colleges’ views on the outcomes of their 
processes on their customers by questioning a sample of learners, to find 
out how well the service met their needs, what worked well and what did 
not work well. We were given permission to use, with small adaptations, 
the student questionnaire used by the FEFC Inspectorate. This (see 
Appendix 2) was sent to a representative group of five of the colleges in 
the survey to circulate to the first 50 distance learners on their current 
records. Replies were sent direct by the students to the LSDA for analysis. 
The response rate was an excellent 31%, allowing us to draw very useful 
conclusions to cross reference with the information from providers. 

Finally, we used the information gathered from the provider visits and the 
learner survey to try to identify the critical success factors in the provision 
of distance learning. The report is based on this. 

The Distance Learning Project, though not a process benchmarking 
project by true definition of the term, utilised many of the principles of 
process benchmarking. 



Process benchmarking is not easy to summarise and, in 
addition, benchmarking is sometimes confused with 
benchmarks - ie performance measures, or with surveys 
It is probably easier to explain benchmarking in terms of 
what it is not. 

It is not Just about comparing standards. 

It is not jjysft about taking part in a survey. 

It is not JysS about league tables of performance. 

It is not JysS about re-engineering your processes. 
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Benchmarking is a systematic method of improvement 
that utilises others’ good practice and learning to improve 
your own processes. It can be used as a way of 
improving any process from ordering paperclips to 
recruiting staff. 

Benchmarking is a method of identifying what must be 
improved in an organisation, finding ways of making 
those improvements and then implementing the 
improvements. 

It requires an organisation to fully understand its 
processes and its customers’ and stakeholders’ needs. 
From that point it is possible to identify gaps between 
needs and performance. 

(Owen 1999) 



We had also intended to carry out desk-based research into the ISR and 
other FEFC/LSC data on distance learning as a basis for comparison with 
our results, but we discovered that the LSC did not currently process data 
for distance learning provision separately as a standard procedure, 
although it could be done as a special exercise. As our investigation went 
on we experienced many colleges’ difficulties in providing us with hard 
data on retention and achievement rates for their distance learning 
provision, and some inconsistencies in their data. 

However, hard data clearly emerges as a significant issue for future policy 
and planning for distance learning. We later recommend to the LSC that 
in future data on open and distance learning is recorded separately from 
full- and part-time provision, and in a way that allows differentiation 



between open and distance learning. This point is also linked to our 
recommendation that the LSC reconsider the definition of distance 
learning in the Funding guidance. 



Our study, although small, included a wide range of distance learning 
schemes. Rural colleges, urban colleges, large colleges, small colleges; a 
national corporate provider and a public service organisation (both 
providing job-related distance learning for their own employees); two 
different types of partnership between colleges and an external 
organisation; small distance learning operations and several that enrolled 
many thousands of learners were all included. The first and overriding 
impression that arises from considering all the data in even this small 
study is of the enormous diversity of distance learning provision that is on 
offer. 

Some programmes in the study were employment related, including those 
provided by employers, by colleges for employers, or by colleges in 
partnership with employers. There were general education programmes 
such as GCSE and A-levels; IT courses including CLAIT and the ECDL; 
short sport or leisure-related courses; business-related programmes such 
as accounting and bookkeeping; a wide range of foreign languages; and 
so on. Most providers offered a limited range, some only a few specialist 
courses, but the brochure of one provider listed almost 600 different 
courses varying from animal management to brief language courses in 
Xhosa, Yiddish and Zulu! 

It was useful to our investigation to find such diversity. Many of the 
concerns of the funding bodies in recent times over distance learning 
quality, especially over retention and achievement rates, have been 
particularly focused on a few subjects with high enrolments, such as the 



IT, bookkeeping and security guarding referred to in the FEFC’s report on 
ODL. These have often been concentrated largely on a few providers with 
very high enrolments. We felt that being able to consider such a diverse 
range of distance learning programmes from providers of all sizes in the 
study should help us to make balanced judgements. 

The range of programmes we found showed in microcosm the huge 
potential of distance learning to offer learning opportunities tailored to 
meet the needs of individual learners, groups of learners and/or their 
sponsors. At the same time, the study also showed how difficult colleges 
find it to harmonise and integrate the management of such a diversity of 
individualised programmes into the main college planning, funding and 
management cycle. 

The consequence often seems to be that distance learning is set up as a 
more or less separate or completely separate operation under the college 
umbrella. One college answered the question ‘How does this distance 
learning scheme integrate into the college’s systems?’ by saying ‘It 
doesn’t’. Several colleges had more than one distance learning scheme 
separately administered and managed. One college had three quite 
separate and different distance learning schemes run in isolation from 
each other (and to some degree also at arm’s length from the college’s 
planning and quality assurance processes). That is not to say that quality 
was not assured, as one of the schemes had a Beacon Award. But it 
would suggest that the college could have problems in monitoring, 
reviewing and developing the schemes successfully in the longer term. It 
was often difficult to locate the ‘process owner’ of the distance learning 
operation in a college. 

The more or less separate arrangements often seemed to lead to 
difficulties and we believe it would be hard to build a significant expansion 
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of distance learning onto some of the existing management and 
administrative arrangements we found. In answer to the question ‘What 
performance indicators do you use?’ the answer from almost all colleges 
as a minimum was enrolment, retention and achievement rates. However, 
the answers to the follow-up question ‘How do you collect data for the 
Pis?’ did not always give confidence about the robustness of the process 
or the data it would produce. Typical answers to the follow-up question 
were ‘Information on enrolment, contact and completion is logged by the 
Programme Leader’ or ‘spot checks’. 

In contrast, there were also college providers who had long-standing and 
successful distance learning operations integrated into their main college 
processes, showing that integration is possible. It is perhaps significant 
that these colleges could provide us with the data we asked for on 
distance learning. 

It seemed from the evidence that to be successful in the long term 
distance learning programmes needed to be included integrally in the 
institution’s strategic planning process and the systems that support this 
process. 



The capacity of distance learning to offer an alternative mode of learning 
to those who cannot or who do not wish to follow attendance-based 
programmes clearly gives it a vital role to play in widening access and in 
meeting the needs of employers for flexible and accessible learning 
opportunities for their employees. 

In answer to the question to learners ‘What are your reasons for choosing 
supported distance learning for achieving your qualification?’ 36% said it 
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was because they could choose their own time to study, 20% because of 
family commitments, and 19% because it was the only way to fit study into 
work. But the additional comments given by learners bring the 
percentages to life: 

Because it was the first time I have studied in 20 years and at the 
time I couldn’t see another way. 

I only undertook the course to get an exam pass quickly (12 

weeks). 

I am a full-time university student and need a grade C in GCSE 
Mathematics to undertake a PGCE. 

Disability makes college access difficult. 

I work 12 hour shifts so distance learning is my only option. 

Agoraphobic - wouldn’t cope well at college. 

Unable to continue daytime class owing to hospital admission and 

recuperation. 

Disabled - cant attend regular classes. 

Did not wish to travel to college. 

Working on contracts in different countries Zambia, Dubai etc. 

The unique capability of distance learning to meet the needs of learners 
such as these makes it an extremely important strand of provision in the 
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post-16 sector. But only high quality distance learning really meets 
learners’ needs. To exploit the full potential that distance learning offers 
we need to know more about the processes that ensure quality in a 
distance learning service. 

Our questionnaire to providers asked about their processes under seven 
headings: planning, financial resources, staffing resources, learning 
materials, equipment and premises, delivery and support systems, and 
customer satisfaction. We were interested in the whole process of 
delivering distance learning from the stage of identifying the market, to the 
design, delivery and costing of the provision, to the exit of successful 
students from their programmes. 

We were searching in all our evidence from these organisations for 
patterns: patterns in their processes for setting up and managing distance 
learning that seemed to suggest a correlation between their processes and 
the outcomes, either to ensure quality for the organisation and the learner 
or not. We believe that we did find evidence to support some conclusions 
on this. 

PflaiminiDinig, momtorimg sumd maimagemoinift 

It would not surprise those involved in providing high quality distance 
learning to be told that overall our research strongly suggests that there is 
a correlation between good quality planning, resourcing and supporting of 
distance learning programmes and the successful outcomes that the 
learners and the organisation itself achieve. 
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Indeed, from our evidence, good practice for distance learning probably 
has much in common with good practice for any course delivered through 
other modes of attendance. For example, the providers who achieved both 
high retention and achievement rates for distance learning were those who 
had identified their target market clearly, had planned the course to meet 
the requirements of their target learners, had a clear specification of the 
requirements for successful study on their programmes, had effective 
support from a range of staff, and had good costing information, efficient 
monitoring and review procedures, and so on. 



...from our evidence, good practice for 
distance learning probably has much in 
common with good practice for any 
course delivered through other modes of 
attendance. 



We also found some poor practice in the study, and much lower retention 
and achievement rates associated with it, which seemed to support the 
evidence of a correlation between processes and outcomes. One distance 
learning operation in this category was typical in not having identified a 
target market carefully. One answer to the question ‘How do you identify 
the target population?’ was: 

We don’t target any group of learners - we just advertise it in the 

prospectus. 



Another replied: 



There is a general understanding that distance learning is more 
suitable for adult learners and will play a part in achieving the 
college’s strategic objectives for widening participation. 

Contrast this with another college with high retention and achievement in 
distance learning provision that answered: 

Market analysis based on past experience and historical trends + 
primary data collected by the sales and marketing team. We then 
target market sector by sector. 




off 



procsssos 



One common pattern we found was that those providers who achieved 
high quality outcomes regarded planning, monitoring, managing and 
resourcing as related not separate processes, and they operated them in 
an integrated way. They understood their own processes, had them well 
documented and could offer documentary evidence of their operation. This 
would appear to be a critical success factor for the provision of a high 
quality distance learning service to learners. Our evidence suggests that 
distance learning that does not have an integrated set of processes for its 
design, delivery and management cannot form a platform for significant 
expansion. 



...providers who achieved high qmiaiity 
outcomes regarded planning, mowitorag , 
managing and resourcing as related not 
separate processes, and they operated 
them in an integrated way. 



Some less successful providers had well developed processes to ensure 
quality in some areas, or had part but not the whole of a process that 
seemed to work well. They did not seem to regard each process as part of 
an integrated whole and often could offer only very limited documentary 
evidence of their processes. A frequent example related to the selection 
and use of learning materials. Most providers had, or said they had, 
systems in place to select learning materials initially against quality criteria, 
but only a minority had follow-up systems to ask for, record and, very 
importantly, to use learner or tutor feedback on the materials. By contrast, 
providers with the most successful outcomes did have such feedback 
loops in place. Similar examples of only partial achievement of quality 
through not ‘closing the loop’ could be found in many of the processes of 
less successful providers. 



One possible result of distance learning operations sitting at arm’s length 
from the main college systems for planning, costing and management is 
that many college providers did not have a clear idea of the costs of their 
distance learning operation. This endorsed the FEFC ODL report view that 
‘Colleges rarely have accurate information on the costs of distance 
learning’ and ‘...it is impossible for most colleges to engage in any 
meaningful cost-benefit analysis’. We found that where distance learning 
is a separate operation it is often managed by a middle manager. Some 
seemed to believe that someone somewhere else might know the costs 
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but they did not (and sometimes were not keen to enquire too closely into 
them for fear of what might be revealed in a cost-conscious climate). 



...only those who planned well knew what 
distance learning cost them. 



The problem did not appear to us to be so much with funding as with 
planning - only those who planned well knew what distance learning cost 
them. One provider with high retention and achievement rates that also 
knew its costs said: 

We spend a lot of time planning and monitoring. 

In some cases a college had a commitment to distance learning as part of 
its widening participation efforts and the costs were reportedly not a major 
concern. It would still seem to us to have been useful to establish them in 
order to manage, and improve and develop the provision cost effectively in 
the longer term, regardless of the separate decisions that the college may 
make about pricing. 

If providers of publicly funded distance learning are to move to a 
guaranteed level of customer service enshrined in a customer charter, as 
we would recommend and which BAOL already requires for the award of 
its Quality Mark, it would be difficult to draw up such a specification if the 
costs of providing the service are not known. This vagueness about costs 
contrasts with the answer from one provider that provides excellent 
support for learners who said: 

We know everything about our costs. 
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and with another where all aspects of each distance learning course it 
offers are costed separately according to the precise method of delivery. 
These costs are recorded in a spreadsheet and monitored monthly, and 
the costing model is reviewed annually. Perhaps predictably, the 
commercial organisation included in the study had very stringent 
procedures in place for the initial costing, the routine monitoring of costs, 
and the longer term financial planning for its distance learning operation. 



The point on which there was probably the greatest divergence in the 
whole survey was on the prices paid by the learners for their course. Most 
learning materials used were produced by commercial providers who 
presumably charge what they believe the market will bear, in order to 
recoup what are often the considerable costs of developing and updating 
distance learning materials. It was still very surprising to find that within the 
college sector all the following charges were quoted by learners as the 
cost to them of a single GCSE subject by distance learning: no charge, £5, 
£25, £33, £90, £100, £120, £170, £200 and £265. 

Even allowing for the fact that some of these rates would be 
concessionary, that the amount of support provided might differ, or that 
learning materials might be included or charged for separately, this is 
nevertheless potentially a very confusing variation for the learner. The 
higher charges also disadvantage learners who may need to learn by 
distance learning but could not afford the high prices charged by some 
colleges. 

The great variation may be in part caused by colleges’ lack of information 
on the costs of their distance learning provision. It should be considered 
whether this is a consequence or a cause of the provision so often not 
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being integrated into standard college processes for setting up and 
managing provision. Some recommended methods by the LSC for building 
up of the costs of a distance learning course for publicly funded providers 
would seem to be timely. For example, one college reported having to 
include a standard charge for college overheads in the costing for distance 
learning courses. This was a charge intended to reflect the use of 
accommodation, and central services such as libraries and computers, yet 
our research showed that over 90% of distance learners used such 
facilities ‘occasionally or never’. 






From our study it would seem that tutor and other contact with the learner 
is a critical success factor in distance learning. All providers offered some 
tutor contact but that in itself is not sufficient. More subtle characteristics 
about the contact seemed to emerge as crucial. These included: 

o speed of the initial contact 

o the close monitoring of students’ progress by the organisation via the 
tutor or administrator 

o speedy and effective follow-up using a range of strategies if the 
learner’s progress appears to be slackening 
o intervention if all appears not to be well. 

One provider with high retention and achievement rates summed up its 
approach to support as being a surrogate parent. More than one provider 
with high retention rates said it constantly monitors learners’ progress and 
one said: 

...if time is running out on an important assignment we chase it up. 
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All providers with high retention rates had well developed and well 
managed processes for providing contact and support. 

From the learner survey we learnt that overall 57% of learners had 
received information about a named tutor contact in a week or less from 
enrolment, leaving 43% who did not. Interestingly, 69% of the learners 
who replied from one provider had heard in a week or less, compared with 
only 36% and 32% of learners from two providers with much lower 
retention rates. It seems that prompt initial contact with the learner within a 
specified time should form one of the process benchmarks for distance 
learning. 

When asked how effective the tutor support had been 71% of all learners 
replied that it had been ‘effective’ or ‘very effective’. The figure for one 
provider with high retention rates was 86%. When asked what had been 
particularly effective the replies from learners from a range of colleges 
noted: 



He replies very quickly. 

Tutor very friendly and enthusiastic. 

...kept in regular contact enquiring about my progress and giving 
me the option to contact the college if needed. 
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When I phone the college distance learning coordinators they are 
very friendly and very helpful. 

I can phone my tutor at home. She is extremely conscientious and 
is in regular contact by letter and phone. 

Contact with my tutor. He is an excellent communicator... 

Encouraging comments from my tutor. 

Although there were many fewer negative than positive comments about 
the tutor support, these included: 

My tutor would only contact me if I sent her an assignment. She 
never asked how I was doing. 

I found the meetings with my tutor were just a means of arranging 
to sit the assessments in a controlled environment. 

The tutor has not been helpful. I have had to rely on friends for 

help. 

There is little doubt from learners’ replies that effective tutor support is 
regarded as crucial to their success. 

Two providers had in the recent past successfully addressed low retention 
rates by improving their learner support arrangements. One said: 

Retention has increased by over 20% on certain courses since we 
offered additional tutor support and workshops, additional prompts 
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and timetable help, extended hours of opening and website help 

etc. 

This good practice contrasts with other providers in the study who, 
although providing some tutor support, apparently had no procedures in 
place for telephone or face to face induction. Some had very complex and 
attenuated procedures, or had no apparent procedures at all, for following 
up learners who missed a tutorial or assignment. Other providers had 
unreliable processes for supplementing tutor contact through helplines or 
contact with administrative staff. 

The difficulties in ensuring effective learner/organisation contact in 
distance learning are clear, but our study found plenty of evidence of 
successful and imaginative good practice. One college programme that 
delivered the majority of a course online, communicated with learners via 
the web, had a PowerPoint presentation about the course accessible to 
them via the web as part of their induction, and had a wide range of client 
support strategies including e-mail, telephone and some face to face 
support. It was a relatively new course, but retention and achievement 
were both 100% to date. 

Interestingly, the tutor contact with distance learners as a whole was not 
particularly ‘high tech’: 68% of learners said that telephone was the ‘most 
frequent’ or ‘next most frequent’ way of contacting their tutor, 59% said 
letter, and only 13% said e-mail. Replies from different providers reflected 
the design of their provision and the support arrangements built into it. 
One provider built in some face to face support, which was the first or 
second most frequent means of contact for 67% of its learners. 

The importance of tutor or other staff contact to the learner is clearly 
magnified by the fact that only 10% of learners in the learner survey are in 
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touch with other learners, and of that small number the vast majority were 
in telephone contact only. Nor do they have much physical contact with the 
college itself as our question on their use of college facilities showed. The 
tutor is the college/provider for the majority of distance learners. 



Providers who maintained a high retention rate for distance learning 
students had well developed systems not only for supporting learners but 
also for identifying individual learning needs, including any needs learners 
had for extra support. This included providers who worked extensively with 
employers. One said: 

The tutor might identify literacy/dyslexia needs - they would refer 

to the in-company contact and together work out a solution. 

OnifiaO ilmtformsiftDoin) sumd guodamxee 

It was surprising to find that only 48% of all learners said they had 
received their initial course information in a week or less. This went up to 
75% in one provider with high retention rates. The remaining 25% of this 
provider’s learners received it within 2 weeks. Among all learners, 28% 
waited between 1 and 2 weeks and a further 24% waited more than 2 
weeks. When one of the potential benefits of a roll-on roll-off distance 
learning course is the ability to enrol at the point of highest motivation, 
having to wait 2 weeks or more for initial information about the course 
cannot be helpful. We need to consider whether more potential learners 
decided not to proceed after such a wait. Prompt response to enquiries for 
information (eg in 2 working days) should be considered an important 
standard for distance learning. 



The tutor is the coliege/provider for the 
majority of distance learners. 
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Prompt response to enquiries for 
information (eg in 2 working days) should 
be considered an important standard for 
distance learning. 



Several providers with high retention rates invested considerable 
resources in the early stages of advice, guidance, enrolment and 
induction, believing that this paid for itself by contributing significantly to 
high retention. In response to the question ‘Did you receive guidance on 
the most appropriate course to take?’ only 46% of all learners answered 
‘Yes’, a serious omission given the funding body requirements on initial 
guidance, quite apart from the need to ensure that distance learners, in 
particular, are making the right choice. Significantly, 86% of learners who 
were enrolled with a provider with a high retention rate said they had 
received guidance. Of all learners who received guidance, 91% said that it 
had been helpful or very helpful, leaving little doubt over the desirability of 
this procedure as far as learners are concerned. Evidence of the need for 
standards on this issue would seem clear. 



Several providers with high retention 
rates invested considerable resources in 
the early stages of advice, guidance, 
enrolment and induction, believing that 
this paid for itself by contributing 
significantly to high retention. 



Interestingly, given the much-quoted potential that distance learning offers 
to individual learners to ‘learn at their own pace', we found that several 
providers with high retention rates specified quite clearly a period for 
completion of the course, and worked hard through monitoring the 
learners’ progress and contact with them, to ensure that they did complete 
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and achieve in that time. The overall picture emerging from the learner 
questionnaire showed that only 8% of learners had been advised that their 
course would take between 1 and 6 months, compared with 38% who had 
been told it would take a year and 54% over a year to complete. 

This gives food for thought in relation to the possible minimum 
specification for aspects of a distance learning course, and to the 
development of performance indicators around such a specification. We 
believe that completion within a specified period should form part of this 
specification, allowing organisations to set targets for 
completion/achievement and to work to improve them. This would build on 
the recommendations currently given in the Funding guidance (para. 194) 
that: 



...distance learning students should be encouraged and supported 
to achieve the qualification or course in as short a time as possible 
to minimise the possibility of drop-out or non-completion. However, 
in some circumstances, it is accepted that the programme may 
take longer to achieve than if delivered by conventional means. 



...completion within a specified period 
should form part of this specification, 
allowing organisations to set targets for 
completion/achievement and to work to 
improve them. 



Support for distance learners is typically shared between specialist tutors 
and administrative staff. The important role of administrative staff, who are 
often the first point of contact for distance learners, has long been 
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recognised and it was therefore surprising to us that most college 
providers interpreted the questions on staffing as applying solely to tutors. 
The multiple tasks of administrative staff in a distance learning operation 
need to be accurately specified and accurately costed. It would add 
considerably to the costs of distance learning programmes if tutors are 
carrying out tasks which can quite satisfactorily be done by administrative 
support staff. There is no recent template for the role of administrative 
staff in distance learning and we recommend that it would be helpful to 
review this now. 

All providers recognised the importance of the tutor to their operation but 
there seemed to be marked differences in the apparent rigour of the 
selection and management procedures between providers with good 
retention rates and the others. Contrast these responses: 

No analysis has been undertaken of the skills and competencies 
required for distance learning tutors employed by the college ... 
there is a recognition that this needs to be done. 

Tutors must have ‘a feeling’ for distance learning ... this is 
assessed on interview. 



with 



Distance learning staff are employed on the same basis as other 
staff- application, references, interview. 

/T utors must be] specialists in the area of delivery. Tutors are 
trained for teaching online by doing the course themselves. In this 
way they learn how the system works and the student’s view. This 
also provides high quality feedback on teaching and materials. 
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Another quality provider said: 



We send out a tutor information pack. Applicants are interviewed 
and if successful move on to induction. They are mentored by the 
coordinator and have regular reviews and monitoring of their 
tutoring ... We require a minimum of a qualification for teaching 
and degree level in their own subject. We start new tutors with a 
small caseload and see how they work with this methodology. 

Of the learners surveyed from this last provider, 85% said that distance 
learning was ‘effective’ or ‘very effective’, and 79% said they would 
‘probably’ or ‘definitely’ choose to study by distance learning again. 
Compare this with figures of 68% and 71%, respectively, for learners as a 
whole. However, the differences in the numbers who were continuing to 
another course of study (not necessarily distance learning) were not so 
great between providers with the most successful outcomes and others (ie 
86% for the learners of one such provider and 85% for all learners). It 
suggests that the experience of learning, even if less satisfactory than they 
had hoped, had nevertheless inspired learners to carry on. 



...attention to the recruitment, 
management and support of both tutors 
and administrative staff must in our view 
form part of the good practice processes 
for distance learning. 



Given our finding that the provision of support is crucial to the success of 
distance learners, attention to the recruitment, management and support 
of both tutors and administrative staff must in our view form part of the 
good practice processes for distance learning. The well known hazard for 
coordinators of distance learning in colleges of having to accept staff with 
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shortfall on their timetables, whether or not they have the appropriate skills 
to act as distance learning tutors, is likely to militate against success in any 
distance learning operation. 

L@airinioinig mateiriialls 

Both successful and less successful providers had procedures in place for 
the selection of materials, usually involving the assessment of 
commercially produced materials by subject specialist tutors and/or the 
distance learning manager/coordinator. One provider was a long-standing 
producer of distance learning materials itself; another had produced 
materials in-house as part of a development project as nothing suitable 
could be found. A wide range of materials was in use in the providers with 
the most successful outcomes. 

A distinguishing feature of these providers was that they all had 
procedures not only for selecting but also for monitoring the use of 
materials, getting feedback from learners and in some cases tutors as 
well, and for using that feedback as part of their management and review 
process. One provider asked all tutors to work through the distance 
learning course they were to tutor on. 



...[the more successful providers] had 
procedures not only for selecting but also 
for monitoring the use of materials, 
getting feedback from learners and in 
some cases tutors as well, and for using 
that feedback as part of their 
management and review process. 



We were surprised not to find more emphasis by providers on the 
importance of the learning materials in their answers, but perhaps to 



established providers of distance learning their importance was self- 
evident. 

Of all learners, 33% said that the learning materials used were ‘not at all 
effective’ or only ‘reasonably effective’. Although only 28% of learners had 
been shown examples of the course material that they were going to use, 
76% of those who had been shown it said that it was ‘helpful’ or ‘very 
helpful’ in making their decision to take the course. Compare this with one 
provider, 54% of whose learners had the opportunity to see the learning 
materials in advance - 86% of its learners said the learning materials used 
were ‘effective’ or ‘very effective’. An opportunity for learners to see and 
assess for themselves a sample of the learning materials to be used would 
appear to be an important example of good practice for distance learning. 



An opportunity for learners to see and 
assess for themselves a sample of the 
learning materials to be used would 
appear to be an important example of 
good practice for distance learning. 



An interesting point made by one provider was that the onerous task of 
ensuring that large stocks of learning and support materials are regularly 
reviewed and kept up to date is a task needing librarian skills, which are 
often not available to a distance learning operation. It is worth noting that 
some suppliers of learning materials offer subscription schemes with 
services which do provide information, advice and regular updates of 
materials and curriculum information, and this can help to minimise the 
problem. 

Another observation from one college was that in some poor quality 
distance learning materials there is a mismatch between the level of the 
qualification at which they are aimed and the level of language use and 
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study skills that they demand for their use. This can lead to low retention 
for that programme. This only serves to emphasise the central importance 
of high quality distance learning materials to ensure maximum levels of 
retention and achievement. Such high quality materials would now include 
guidance to learners over study methods, in order to meet a college’s 
statutory health and safety obligations. 



The questions on premises and equipment elicited a range of answers 
from providers, with few patterns emerging. It would seem to us from the 
answers received that for a distance learning provider premises are, as 
might be expected, not usually a major issue. Some distance learning 
courses still involve face to face contact for tutorials, and appropriate 
accommodation is needed and was mentioned by providers who offered 
this service. One rural college allowed tutorials to take place in a learner’s 
or tutor’s house. 

Information from the learner survey would bear out the low importance 
generally accorded to the premises by providers. In answer to the question 
about use of the college facilities, the computers were the most used 
facility, but even here only 7% of the learners used them weekly, and 93% 
‘never’ or ‘occasionally’. A similar picture emerged with library (99% 
occasionally or never), e-mail (94% occasionally or never) and the other 
facilities listed in the question. This is despite the fact that only 34% of 
learners said that access to the facilities was ‘difficult’ or ‘very difficult’. Of 
the learners who answered ‘difficult’ or ‘very difficult’, 95% of them said 
that it was difficult or very difficult because of distance (perhaps 
unsurprising for a distance learning course). 
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Equipment, usually interpreted by providers as ICT equipment, was seen 
as necessary to their operation by all providers, though some were 
planning to make, or were already making, more use than others of the 
potential for web-based communication. Again there were no clear 
patterns evident to distinguish successful from less successful providers. 



Providers were asked about the systems they had in place to obtain 
regularly the views of clients, learners and sponsors, whether or not they 
had a complaints policy, and how they used customer feedback to review 
and develop their service. 

All providers had some procedure in place to obtain customer feedback, 
but once again the differences between providers with high retention rates 
and others was apparent. One provider with little statistical information 
available on retention and achievement said: 

After 1 month students are sent a feedback form. There are no 

further on-course questionnaires. 

Most college providers seemed to use the college systems for surveying 
all students’ satisfaction via ‘student perception of courses’ questionnaires, 
or ‘quality’ questionnaires. All said they had a customer charter; some 
used the college charter only and others had a separate one for their 
distance learning operation. All had a complaints policy/procedure, often 
set out in a student or customer handbook. However, one provider made 
the point that a college charter often focuses attention on access to 
college facilities and services not relevant to distance learners. This 
suggests that a charter specific to distance learners should be 
recommended. 
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What distinguished! the providers with the 
most successful outcomes from the 
others was the use they made of the 
survey results, feedback from learners or 
the record of complaints. 



What distinguished the providers with the most successful outcomes from 
the others was the use they made of the survey results, feedback from 
learners or the record of complaints. For example, they had a clear line of 
responsibility that they quoted for dealing with a complaint. One said: 

Course Specialist, then Line Manager, then Head of Unit - and all 
complaints notified to HOU. Customer feedback is raised in all 
internal meetings. 

Another provider of quality services said: 

We have a complaints procedure with an agreed turn round time 
and each complaint logged. We incorporate customer feedback 
through an analysis of student surveys and of complaints into 
Programme Reviews. It triggers an action planning process that 
resolves complaints, but we try to resolve issues before they 
become complaints. 

It was probably no coincidence that we found that all providers with high 
retention rates also had one or more customer care awards. Interestingly, 
from the learner survey we found that only 67% of learners overall said 
they had been given an opportunity to comment on the efficiency and 
effectiveness of the course programme, although that rose to 91% for one 
of the providers with high retention rates. Despite the attention paid to 
getting learners’ views by all providers, only 20% of those who had been 
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surveyed said that they in turn had received feedback on the comments 
that they had made. 

It would seem that there is evidence of the need for each learning provider 
to produce a defined process. This should not just solicit customer opinion 
but should be seen as a vital source of information for the development of 
a quality service and for use in the management process. 



The allegation of poor retention and achievement rates has long been 
made by the critics of distance learning. There is plenty of evidence to 
support this allegation, including that offered in the FEFC report on ODL. 
That report stated: 

Achievement rates are very low. Retention rates are low, though 
hard to measure accurately because colleges are not always 
certain whether students are still active learners or have withdrawn 
from their courses. 

However, we believe that our study supports the view, offered many times 
in defence of distance learning, that high quality programmes have 
retention and achievement rates that compare very favourably with 
retention and achievement rates on full- or part-time versions of the same 
programme. Among colleges in the study that had high quality processes, 
in most or all areas retention rates were high. For example, one provider 
with robust data and several quality awards had retention of over 90% for 
the past 3 years, and retention to date was 100% on individual courses at 
two other colleges. Achievement rates in one college were over 80% for 
the past 3 years. These rates compare very favourably with the rate of 
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84.1% for adults on part-time programmes in general FE colleges in 
1999/2000 (LSC 2001). 



...high quality programmes have retention 
and achievement rates that compare very 
favourably with retention and 
achievement rates on full - or part-time 
versions of the same programme. 



Interestingly, high retention was not always accompanied by high 
achievement rates. In some cases there was an apparent anomaly 
between high retention rates and much lower achievement rates (eg 
retention of 86% and achievement of 36%). We believe this pattern merits 
further investigation. Are learners continuing to stay on the programmes 
because they are getting the outcomes they want from them, even though 
these may not be formal qualifications? Do they complete the course but 
not enter for the qualification? Are they attempting the qualifications and 
failing? Are the statistics telling the whole story about learners who may 
take longer to achieve their qualifications than learners via other modes? 
We believe some further investigation of patterns where retention is high 
and achievement much lower is urgent if rational planning decisions are to 
be made about distance learning in the future. 



We be ft eve some further investigation of 
patterns where retention is high and 
achievement much lower is urgent if 
rational planning decisions are to be 
made about distance learning in the 
future . 



Several possible contributory factors were mentioned to us. For example, 
an employer paying for an employee on a distance learning course may 



not be interested whether or not the employee gets a qualification. Also, 
some learners may want to complete only some units or modules of a 
programme, which currently would not contribute to achievement data 
although the learner has achieved the outcome they wanted. 

Sometimes high retention rates for distance learning are queried - are 
colleges tracking learners effectively and so is the data on who is or is not 
still on the programmes reliable? Our experience of asking for retention 
and achievement data and finding that not all colleges could provide it 
routinely suggests that in some cases this could be a valid question to ask 
about an apparently high retention rate. But we also found the anomaly 
between high retention and much lower achievement rates for 
programmes where good data was available and very robust processes to 
assure quality were in place. Unreliable retention data is not in our view 
the whole story. Several colleges noted to us the difficulty of collecting 
data for distance learners who may start and complete their programme at 
any time of the year, when the main college management information 
system is set up for a different cycle. It is unfortunate, if unsurprising, that 
many distance learning coordinators try to keep their own data, which is 
then very difficult to validate against centrally held data. 

A view expressed by several colleges is that retention rates for distance 
learning courses are lower for learners who do not pay fees and that the 
level of course is also influential: also that when a high proportion of 
learners’ fees are being paid for by their employers retention is high. 
These views merit further investigation. 

Perhaps the consequences of poor practice are particularly exposed in 
distance learning programmes because there are few other social factors 
to hold learners and compensate for lack of quality in course design or 
delivery. For example, peer contact and contact with an institution may 





45 



hold students on an attendance-based programme even when they are 
not very satisfied with it overall. We would suggest that the lack of these 
cohesive factors should perhaps explain to funding bodies retention and 
achievement rates for distance learning which do not necessarily match 
those of attendance-based programmes. This does not suggest that rates 
significantly lower than attendance-based programmes are acceptable. 
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One strength of the distance learning method is the enormous potential it 
gives to providers to vary the design of the programmes to meet the needs 
of learners or their sponsors. Varying amounts of tutorial time and home- 
or work-based study can be combined, if required, with some attendance 
for group tutorials, occasional classes or even residential study. The length 
of the programmes can be fixed or variable. The advent of new 
technologies in recent years, culminating in online learning and the 
potential offered by the web, has increased the possibilities for 
individualising programmes in truly mind-blowing ways. 

If the programme is being costed accurately at full economic rates and 
paid for by the client, be they individual or corporate, then the exact design 
of the programme does not cause a problem. No one is worried about 
deciding whether it is or is not defined as ‘distance learning’. The law of 
the market will to a large extent decide whether the quality and value for 
money satisfies the customer. 

However, if distance learning is in whole or part publicly funded the 
funding bodies need to ensure that the programmes offer quality and value 
for money. They have a legitimate interest in the specification and delivery 
of the programme. 

The apparent difficulties that many colleges have in integrating a range of 
different distance learning programmes into their standard college 
management processes is a reflection in part of a tension that has 
developed. This tension is between the providers’ wish to exploit the 



potential of the distance learning method to meet learners’ needs for 
flexibility in their learning opportunities, and the funding bodies’ need to 
ensure quality and value for money in publicly funded distance learning. 
This tension has given rise in recent times to ever more elaborate 
definitions of distance learning in the Funding guidance. Our survey would 
suggest that the definitions and associated regulations are now in danger 
of becoming counter-productive, in discouraging providers of good quality 
distance learning as well as excluding poor distance learning provision. 

An established provider of large-scale distance learning that would meet 
all quality criteria noted: 

The funding tariff is extremely problematic. It is not clear what 
funding will accrue from different provision and there are no 
definitive answers from the funding bodies (FEFC or LSC). There 
is a great deal of ambiguity - definitions seem to shift and different 
auditors have varying opinions on what is acceptable. This 
hampers business activity and planning. In fact, this year we have 
drawn back from distance learning as it is so complicated to work 
out the funding and we are not confident that provision will draw 
down the funding anticipated. We will be more likely to offer a 
mixed mode of learning, though the difference between this and 
distance learning is largely a matter of interpretation. We wish to 
emphasise the urgent need for clarity in terms of definitions and 
funding guidelines for distance learning. 

This tension is undoubtedly exacerbated by many colleges’ inability to 
identify accurately the costs of their distance learning provision. The 
provider who made the above comments had very accurate information on 
costs. However, for many colleges the combined effect of a complex LSC 
definition, almost infinite variations of distance learning course design, and 
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their own difficulty in identifying and tracking costs will act as a powerful 
deterrent to the expansion of distance learning, because of the potential 
risks it seems to pose to the college. No wonder the ‘solution’ has often 
been to leave distance learning as minority provision at arm’s length from 
the college systems. 

We began to ask, perhaps surprisingly, whether the funding bodies’ 
definitions are trying to be too all-embracing? It might be more 
manageable for both funding bodies and providers to have a clear 
framework for fundable distance learning programmes, perhaps drawing 
on the work of Mick Fletcher on the taxonomy of open and distance 
learning (Fletcher 2001). We hope that beginning to define good practice 
processes for distance learning through this study may help the LSC’s 
consideration of this. 

We believe that the LSC should consider as interlinked issues to be 
addressed together in relation to publicly funded distance learning: 

o a minimum specification or series of specifications for the design of 
distance learning programmes (together with recommendations on 
targets) 

o guidance on a costing method 
o a review of data collection 

o recommendations on good practice for distance learning. 

Recommendations on good practice processes in distance learning 
operations could also perhaps be helpful to the Adult Learning 
Inspectorate. 
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Learning to succeed (DfEE 1999) sets the clear imperative The interests 
of the learner must come first’. 

This has now been enshrined in the new Common Inspection Framework. 
The CIF makes it clear that systems and processes are of interest only to 
the degree that they help to ensure a high quality experience for the 
learner. Providers will be judged on their ability to plan and deliver a quality 
learning experience effectively and efficiently. 

The requirements of the CIF, and the parallel requirements of the funding 
bodies, will apply to distance learning provision the same as to any other 
provision - there will be no special cases. These requirements include the 
expectation that realistic targets will be set for learner retention, 
achievement and completion; that providers will collect and produce robust 
data; that learner feedback will be obtained and used; that evidence-based 
self-assessment will take place; that the results will be used in an 
integrated way in the organisation’s cycle of planning, management and 
review; and so on. 

The results of our study have shown that it is possible for colleges to meet 
these requirements for their distance learning operation but that at present 
not all do so. In defence of colleges, it must be said that currently there are 
significant challenges to be overcome before they can do so. In our view, 
many arise from the tension between the almost limitless permutations 
that distance learning offers to providers for course design, start and 
completion dates etc, which have to be set against necessarily prescribed 
systems of the funding bodies for all publicly funded provision, which must 
then be enshrined in definitions and regulations that apply to distance 
learning. 
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It is no wonder that many colleges put what they see as the complications 
and even potential risks of running distance learning at arm’s length from 
their main systems. This can create the self-fulfilling prophecy: 

Distance learning is seen as difficult to run -» -» 

-)>-)> it is put outside the standard college systems with often 
inadequate alternative systems and support in place — » — » 

-» -» the operation may run satisfactorily on a small scale but 
runs into difficulty over monitoring and management if it tries 
to expand because the infrastructure is not there -» -» 

-» -»it either stays small and fairly marginalised or may become the 
huge ‘black hole’ that no one can produce robust data for -» -» 

-> -> it is highly likely then to be seen as risky and may well 
be discontinued. 

We have found that distance learning can be run cost effectively within 
college systems with high retention and achievement rates and high levels 
of customer satisfaction. This requires customers’ needs to be seen as 
central, and key processes which we have identified to be in place and 
managed in an integrated way. Our specific recommendations follow. We 
hope that these will form a significant practical contribution to the future 
planning of distance learning so that it may reach its true potential in the 
new post- 16 sector. 
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[^©©©mmeimdlaflbims to to© LSC 



The first set of recommendations is for the LSC. We recommend that it 
consider supporting colleges by: 

© setting a minimum specification or small series of specifications for 
funded distance learning provision, rather than having a single wide- 
ranging definition which is open to different interpretations, as at 
present 

© issuing some guidance to sector providers on a costing method for 
distance learning 

© recommending some parameters for charging policies for distance 
learning, in particular in relation to the inclusion of the cost of learning 
materials in the charge quoted to students 
© revising the criteria for data collection on open and distance learning to 
establish a category of data for distance learning separate from part- 
time, and to allow differentiation between open and distance learning 
© giving guidance to colleges on the tracking of distance learners and on 
the collection of tracking data for distance learning students 
© recommending process benchmarks and associated performance 
indicators for distance learning based around those we suggest 
© putting in hand work on the collection of statistical information on 
distance learning to allow the establishment of metric benchmarks for 
distance learning at an early date 

© investigating further the apparent anomaly between high retention and 
lower achievement rates in distance learning 
© considering issuing good practice guidelines for the operation of 
distance learning in the post-16 sector. 
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IRecommemidlsifiDomis to providers 

The second set of recommendations is to providers and relates to their 
management of distance learning provision. Our study suggests that they 
need to: 

o have clear procedures for establishing and reviewing regularly the 
costs of all aspects of their distance learning courses, whatever 
decisions may be made later about pricing 
o offer effective initial guidance to all distance learners 
o specify realistic times for the completion of a distance learning course 
and put in place systems to support learners in their achievement of 
them wherever possible 

o set up robust procedures for tracking and monitoring learner/tutor 
contact with recognised criteria and procedures for intervention where 
necessary 

o establish recognised and rigorous procedures for recruiting and 
managing tutors and administrative staff and specifying the skills and 
competences for those staff involved in supporting distance learning 
o set up systems to obtain and use feedback from distance learners on 
their experience of their course, including their tutor support, and their 
views on the learning materials used 
o set targets for aspects of customer service, such as responding to an 
enquiry, referral to a tutor etc, and monitor achievement of them 
o use the quantitative and qualitative data from their monitoring 
processes routinely to improve service levels 
o integrate distance learning provision into the standard college 

processes for planning, costing, management, monitoring, review and 
evaluation. 
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Learner imeedls 

The third set of recommendations, again for providers, is based on the 
belief that the starting point for their distance learning provision should be 
the identified needs of their customers (learners and sponsors). 

We recommend a customer charter with a guaranteed level of service 
including a commitment to: 

° systematically investigate the needs for distance learning provision in 
their locality (given that many local learners cannot access class-based 
college provision) 

° design distance learning provision to meet identified needs 
° respond within given times to an initial request for information 
o offer impartial and effective initial advice and guidance to all potential 
distance learners, including the consideration of modes other than 
distance learning courses 

° give customers the opportunity to see a sample of learning materials 
that would be used for their course 

° guarantee times for first contact with their tutor after enrolment and 
times for receipt of learning materials after enrolment 
o specify the amount and nature of tutor contact 
o specify the access to facilities 

o make arrangements for learners with additional learning needs 
o give advice on progression routes following their distance learning 
course 

o provide information about the complaints procedure with a guaranteed 
response time to complaints 

o give customers the opportunity to give feedback on their distance 
learning course. 
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Appsmidfe 11 OBstaini©© toamoinig pr®¥od®r 
gysstfcGUimaor® amid ba®kgr®y[mdl symmary 

PDaiiniiTiBinig ftlh® s@rFD<s® 



Areas Ddleinfiofiedl as IboSmigi contributory to success: 

° target population is identified 

° goals/aims/objectives are set for distance learning operation and 
individual programmes 

° targets set are related to quality for key aspects of the service 

© systems for collecting data to monitor and review key processes are in 
place (including customer feedback) 

© planning for the service is integrated into corporate planning of the 
organisation. 



Prompts and (questions 


Responses 


How do you identify the 
target population? 




How do you identify 
customer needs? 




How do you set 
goals/aims/objectives for 
DL as a service? 




How do you set 
goals/aims/objectives for 
individual programmes? 




What performance 
indicators do you use? 





How do you collect data 
for the Pis? 




How do you use the data 
for planning and 
reviewing the service? 




How does the DL 
planning and review 
cycle tie in with the 
organisation’s planning 
and review cycle? 





Oocymemite to request 


Avaii liable? 


Flowcharts of processes 




Documented procedures 




Documented 

goals/aims/objectives 




Corporate/departmental 

plans 






58 



57 



R@s®ur<£®§ - fomam©® 

Areas identified as being) contributory to success: 

q budget available and allocated 

© distance learning programmes and overall operation costed 
© financial targets set and monitored. 



Prompts and questions 


Responses 


How do you identify the 
budget requirements for 
DL? 




Who is responsible for 
allocating the budget to 
DL? 




How did you identify the 
costs of delivering the 
programme? 




How do you review these 
costs? 




What financial targets 
are set? 




How are the targets 
monitored? 





59 

o 

ERIC 



58 



Documnerofts to request 


Available? 


Flowcharts of processes 




Documented procedures 




Minutes/report of reviews 




Reports on targets 
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Areas identified as being contributory to success: 

o clear definitions of roles/job descriptions/competences required 
© staff with appropriate mix of skills (ie managerial, admin, tutoring, 
specialist technical) to staff distance learning operation 
© staff development opportunities linked to individual and business 
requirements. 



Prompts and questions 


Responses 


How do you identify the 
competences for DL staff 
and how do they differ 
from non-DL staff 
competences? 




How do you identify staff 

development 

requirements? 





Documents to request 


Available? 


Flowcharts of processes 




Documented procedures 




Job descriptions of DL 
and non-DL staff 




Staff development 
reports/requests 
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mmateroals 



o system in place for identifying quality learning materials in all media to 
support programme(s) offered 

o system in place for purchasing learning materials for stock or learner 
use. 



Prompfe aindl qjuesftooims 


IResporoses 


How do you identify what 
learning materials are 
required? 




How do you obtain 
learning materials of a 
suitable standard? 




Who is responsible for 
purchasing/obtaining 
learning materials? 




How are current learning 
materials reviewed for 
suitability? 




How are learners 
consulted on learning 
materials? 




What use do you make 
of IT/interactive 
materials? 
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DoeymomiSs to request 


Available? 


Flowcharts of processes 




Documented procedures 




Minutes/reports on 
learning material reviews 




Examples of learning 
materials 




Feedback from learners 
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o§@§ 



o premises and equipment needs identified 
o premises meet statutory requirements 
o premises/accommodation fit for purpose 

o appropriate equipment available to meet requirements of learners 
o appropriate technologies provided to meet learners’ 
needs/requirements 

o adequate technical support available for all equipment available. 



Prompts amd qiuesdooms 


Responses 


How do you identify what 
premises are required? 




How do you 

manage/maintain 

premises? 




How do you identify what 
equipment/technology is 
required? 




How do you 

manage/maintain 

equipment/technology? 





Oocomeplts Ho ireqjoesll 


Avaii liable? 


Flowcharts of processes 




Documented procedures 
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Areas iderDfiofoed as beomig eorjfirobutory So success: 

° clear information available to learners/sponsors about services offered 
level and types of programme offered; cost; availability; arrangements 
for tutor support: who, when, where; qualifications that may be 
achieved 

® impartial advice and guidance provided to help potential 

leamers/sponsors clarify their learning needs and the extent to which 
distance learning is appropriate for them and to choose a distance 
learning programme if this method is appropriate 
® systems in place to identify any additional learning needs that learners 
may have 

o systems in place to provide support for any additional learning needs 
identified 

® systems in place to allow learners to demonstrate and record 
achievement/obtain qualifications 
© systems in place to allocate tutor/mentor with appropriate skills 
® systems in place to monitor and record learner progress and 
achievement 

© systems in place to monitor and record performance/caseload of 
tutor/mentor 

© systems in place to identify learners who become inactive against 
agreed criteria, to contact them and agree next steps 
(continuation/withdrawal). 
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Prompts amid qi^esHoomis 


Responses 


What information do you 
give about the services 
and subjects that you 
offer? 




How do potential 
learners obtain 
information and advice 
on the suitability of the 
learning methods, 
subjects, timescale, level 
and qualification? 




How do you induct new 
learners? 




How do you identify 
additional learning 
needs? 




How do you provide for 
those additional learning 
needs? 




How do you identify 
learners’ needs? 




How do you match 
learners’ needs with the 
appropriate tutor/mentor 
skills? 




How do learners 
demonstrate and record 
achievement? 






65 



How is this achievement 
monitored? 




How do you identify 
learners at risk of 
leaving/failing? 




How do you offer support 
and advice to learners at 
risk of leaving/failing? 




How are qualifications 
obtained? 




What strategies have you 
used to improve retention 
and achievement? 





□©©ymemite to recguesfi 


Available? 


Flowcharts of processes 




Documented procedures 




Retention achievement 
reports 




Definitions of 
achievement, retention 
etc 




Induction materials 




Information leaflets 




Retention and 

achievement 

records/reports 
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Areas identified as (being) contributory to success: 

o systems in place to obtain regularly the views of clients, learners, 
sponsors 

° complaints policy/customer charter in place and displayed to customers 
o information from customer feedback used to review and develop 
services. 



Prompts and questions 


Responses 


How do you obtain 
feedback from your 
customers? 




Do you have a customer 
charter? 




What is your complaints 
policy/procedure? 




Who is responsible for 
monitoring complaints? 




How do you inform your 
customers of how to give 
feedback/complain? 




How do you incorporate 
customer feedback into 
service review and 
development? 




Do you have any 
customer care awards? 
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Avail Sable? 


Flowcharts of processes 




Documented procedures 




Customer charter 




Complaints leaflets 
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o distance learning often fails to meet individuals’ needs 
o induction processes are often unsatisfactory 
o students’ needs for additional learning support are often neither 
identified nor met 

o key skills are not developed effectively, assessed nor accredited 
o opportunities for personal support are few 
o the use of IT to support students’ learning is underdeveloped 
o achievement rates are very low 

o retention rates are low, though hard to measure accurately as colleges 
are not always certain whether students are still active or have 
withdrawn 

o quality assurance arrangements have failed to produce improvements 
in retention and achievement 

o inadequate use of target setting and performance indicators 
o links between strategic planning and management of the curriculum 
are weak and curriculum management is poor 
o colleges rarely have accurate information on costs of distance learning. 

Criticisms of open learning provision in the summary of the report 

o overall costs of setting up and running open learning centres both 
within the college and at community venues are rarely carefully 
analysed 



information provided to prospective students is not always of good 
standard 

some receive inadequate advice and guidance when selecting their 
programme of study 

quality of tutor support varies with IT students often receiving the most 
effective support 

students’ needs for additional learning support not always met 
learning materials not always matched to students’ abilities 

colleges slow to modify materials where common difficulties have been 
identified 

technology has not been used effectively to make learning materials 
more interactive. 



° C@ll@g@ imam® D®®™i!oi)g 

amd sCsffls 
(ifteveDcDlfDctrooinrii 

This form will be electronically scanned. Please complete using black ink, ag®m®y 

marking the relevant boxes like this:\/^\. If you make a mistake please 
complete the correct box and mark the incorrect box like this:^ . 

When providing written answers please only write within the space provided. 

Q1 How ©lid are yoy? 

□ 16-19 □ 20-24 □25-35 □ 36-45 □46-55 □over 55 

Q2 Whaft sex are yoy? 



I | Male □ Female 



Q3 Which of due following apply to yoy? (Please mark all that apply.) 



Employed full-time 
Full-time carer 
Unemployed 



Employed part-time 
Part-time carer 
Retired 



Q4 How did yoy discover Shaft disftance learning was a possible way fto sftydy? 





Independently (on my own) 


1 Through friends 






Through careers service j 


j Job centre 





Through college 
Through employer 



Q5 How did yoy conftacft fthe college in ftlhe firsft insftance? 

I I Telephone □ E-mail □ Internet □ Letter □ In person 



Q6 Whaft are yoyr reasons for choosing sypporfted disftance Beaming for achieving yoyr 
qyalificaftion? (Please mark all that apply.) 

Choose own time to study so more convenient for me 

Only way to fit study into work pattern 

Only way to study this particular topic 

| Used this method of study in the past 



Q7 Were any of fthe foBlowing alfternaftive ways fto sftydy discyssed wifth yoy? 

(Please mark all that apply.) 

□ Full-time course Q Part-time during the day 

One day per week □ Evening course 

Combination of part-time day and evening 

Buy tutor time by the hour at a time that suits you 



No course available locally 
Family commitments 
Other (please state below) 



Survey : 70 

^ mu Bin 



Page : 1 71 




Learning and Skills Development Agency 



Q8 a) 



□ Yes □ No (Go to Q9) 

Q8 b) Was SS by: 

□ Telephone □ E-mail □ Internet □ Letter □ In person 

Q8 c) How helpffyl was She gyidance? 

I I Unhelpful Q Not very helpful Q Helpful Q Very helpful 

Q9 a) Were y©y shown examples of She coyirse maSerial yoy are going So yse? 

□ Yes □ No (Go to Q10) 

Q9 b) How helpfyl was Shis ddi making yoyr (decision) So sSydy? 

I I Unhelpful Q Not very helpful Q Helpful Q Very helpful 

Q10 How long was 5S before yoy received coyrse informaSion? 

I | 3 days Q A week Q Two weeks Q Over two weeks 

Q1 1 How long have yoy been Sold yoyr coyrse will Sake? 

I I 1 - 3 months [] 3-6 months Q Up to 1 year Q Over 1 year 

Q12 WhaS is She SiSI® of She coyrse yoy are sSydying? 



Q13 How long was is afSer accepSance on She coyrse (sSydy programme) ShaS yoy received 
informaSion aboyS a named conSacS (SySor)? 

I | 3 days Q A week | | Two weeks Q Over two weeks 



Q14 Which are She Swo meShods yoy mosS freqyenSly yse So conSacS yoyr SySor ? 
(Please mark I for the most frequent, and 2 for the next most frequent) 





Telephone 




E-mail 






Letter 




In person 





Internet 

Video conferencing 



Q15 If She SySor is noS available when conSacSed, whaS alSernaSive adminisSraSive sypporS is offered 
So yoy? (Please mark all that apply.) 



Message taken 
Referred to another tutor 
Asked to call back 
Told tutor will 'phone back 
None of these 
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Weekly □ Every two weeks □ Monthly 

Only when I have a problem □ When I have submitted an assignment/unit 

Other (please specify below) 



Q17 a) Aire you otra ffoyclh wlfflh ©ffber sffy dents sffydylng toe same coyrse (programme)? 

□ Yes □ No (Go to Q18) 

Q17 b) How do you get 5m toy c Bn? 

(Please mark I for the most frequent, 2 for the next most frequent, etc. for each method you use) 





Telephone 




E-mail 






Letter 




In person 





Chatline 

Video conference 



Q18 How often do yoy yse toe following general college facilities? 

Weekly Monthly Occasionally Never 

Library (use printed material) 

Computers for general use 
E-mail 

Internet (for searches) 

Learning materials held electronically 



(including assignments, etc.) 
Multimedia 


□ 


□ 


□ 


□ 


(using learning materials interactively) 


□ 


□ 


□ 


□ 



Q19 How easy os access to toe general college facilities? 

| | Very difficult □ Difficult □ Acceptable □ Easy 

Q20 Iff yoy answered 'dlfffflcylt' or Very dlffffScylt' to QH9, was Iff becayse off: 



It not being part of the contract Yes 
The distance from the college Yes 

The times of opening Yes 



No 

No 

No 



| | Very easy 



Q21 a) Are yoy given opportynitles ffo comment on fflhe efficiency and effectiveness off toe coyrse 
(programme)? 

□ Yes □ No (Go to Q22) 

Q21 b) Have yoy received feedback on toe commentt? 

□Yes QNo 



r~ © 

c ERLC 
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Q22 a) How effective cOIcB you And the following? Not at all Reasonably Effective Very 

effective effective effective 

The learning materials used to support the course I II II — I | — 

The tutor support — — — 

The college support (other than tutor support) — 



Q22 b) Pleas® say what you found to be particularly effective: 



Q22 c) or particularly DinietfffocttDV®: 



Q23 Air© course materials: 

I I Charged separately Q Included with the tuition fees 

Q24 How Is your coyirs© paid for? 

B B y y° u O By your employer [~] I'm exempt from paying 

Through an Individual Learner Account (I LA) 

Q25 What is She cost off your course? i 1 

£ 

Q26 How effective a way off learning have you found distance [©arming? 

ED Not at al1 effective □ Reasonably effective Q Effective Q Very effective 

Q27 Iff you were to take another course off study, would you choose to study by distance learning 
agson? 

□ Definitely not □ Possibly □ Probably □ Definitely 

Q28 Air© yotuj ioteodoirog to confiooy© with airoy cows© {not iro©c©sssiiroly by distoroo© 0©amoinig) a fftoir 
compll©ti5inig this couirs©? 

EDYes Dno 

Q29 Have you received infformation on progression to other courses, either at the college or 
elsewhere? 

EDYes Dno 



Thank you for taking the time to complete this questionnaire. Please return it to : 
Jane Owen , LSDM, FREEPOST (BS6745), London, SE11 5BR 
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The Learning and Skills Development Agency 
(LSDA) is a strategic resource forthe 
development of policy and practice 
in post-16 education and training. 

The National Extension College (NEC) is a 
not-for-profit educational trust dedicated to 
widening educational opportunities for adults 
through distance learning. 
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